Music Information Retrieval from a Singing Voice Based on Verification of Recognized Hypotheses

Several music information retrieval (MIR) systems have been developed which retrieve musical pieces by the user’s singing voice. All of these systems use only melody information for retrieval, although lyrics information is also useful for retrieval. In this paper, we propose an MIR system that uses both melody and lyrics information in the singing voice. The MIR system verifies hypotheses output by a lyrics recognizer from a melodic point of view. Each hypothesis has time alignment information between the singing voice and recognized text, and the boundaries of each note can be estimated using the information. As a result, melody information is extracted from the singing voice. On the other hand, the melody information can be calculated from the musical score of the song because the recognized text must be a part of the lyrics of the song. The hypothesis is verified by calculating the similarity between the two types of melody information. From the experimental results, the verification method increased the retrieval accuracy. Especially, it was very effective when the number of words in the user’s singing voice was small. The proposed method increased the retrieval accuracy from 81.3% to 87.4% when the number of words was only three.